Property control and configuration based on thermal imaging

ABSTRACT

A monitoring system that is configured to monitor a property is disclosed. The monitoring system includes a thermal camera that is configured to generate a thermal image of the property. The monitoring system further includes a monitor control unit that is configured to receive, from the thermal camera, the thermal image. The monitor control unit is further configured to, based on the thermal image, determine a temperature of a portion of the property depicted in the thermal image. The monitor control unit is further configured to determine that the temperature of the portion of the property depicted in the thermal image satisfies a temperature threshold. The monitor control unit is further configured to, based on determining that the temperature of the portion of the property depicted in the thermal image satisfies the temperature threshold, select and perform a monitoring system action.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.16/508,794, filed Jul. 11, 2019, now allowed, which claims the benefitof U.S. Provisional Application Ser. No. 62/696,394, filed Jul. 11,2018. The complete disclosures of all of the above patent applicationsare hereby incorporated by reference in their entirety for all purposes.

TECHNICAL FIELD

This specification generally relates to monitoring systems.

BACKGROUND

Many properties are equipped with monitoring systems that includesensors and connected system components.

SUMMARY

Many residents and homeowners equip their properties with monitoringsystems to enhance the security, safety, or convenience of theirproperties. The property monitoring systems can include one or morethermal imaging sensors, such as an infrared (IR) camera, which providedata related to the temperature of a particular area of the property.For example, an IR camera located in a room of the property may captureimages that can be processed to generate a temperature map of the room,enabling identification of temperature variations with high spatial andthermal granularity. Furthermore, by processing a sequence of IR imagesrecorded over time (e.g., a video), temporal variations in temperaturecan also be identified and quantified.

In some implementations, monitoring systems can dynamically control andconfigure devices and components of the property based on thermalimaging data. For instance, the monitoring system can use the dataprovided by a thermal imaging sensor to determine a temperature of anarea of the property and, based on the determined temperature, detectunexpected or undesired activity on the property. As an example, basedon fine-grained temperature data derived from thermal images of a roomof a home, the monitoring system may be able to detect that an electriciron has been left at an elevated temperature after the resident hasleft the home or that a door to a refrigerator has been left ajar afterthe resident has left the room. The monitoring system can subsequentlyperform an action to alert the resident to the unexpected or undesiredactivity, for example, by sending a notification to the resident'smobile device. In some cases, the monitoring system may take an actionto mitigate the undesired activity, for instance, interrupting powerdelivery to the outlet to which the electric iron is connected in orderto reduce the safety hazard presented by the hot, unattended iron.

In some implementations, the monitoring system may dynamically configureone or more appliances or monitoring system components based on analysisof captured thermal images of the property. For example, the monitoringsystem may process thermal image data to identify a resident of theproperty in the thermal image. The monitoring system may furtherdetermine from the thermal image data that the resident's bodytemperature is higher than usual (which could occur, for example, if theresident had recently engaged in a strenuous activity, such as anexercise workout). If the resident's body temperature is higher than apredetermined threshold, the monitoring system may adjust the thermostatto cool the home and provide a more comfortable environment for theresident.

Certain implementations of the disclosed systems, techniques, andmethods have particular advantages. In some cases, by analyzing thermalimage data, a monitoring system can detect unexpected or undesiredactivity at a property and perform actions to mitigate or preventfurtherance of the activity. For example, based on thermal image data,the system can detect overheated appliances or other fixtures that posea fire hazard and notify a property resident of the hazard. In somecases, the monitoring system may be able to improve the efficiency oroperation of the appliances of a property based on thermal image data,for example, by detecting appliances that are hotter or colder thanusual, which may indicate inefficient operation or malfunction. In somecases, the monitoring system can use thermal image data to detect localaberrations in room temperature that indicate a region or source ofunusually hot or cold air (e.g., a window that is open while the airconditioning or heat is operating). Upon detection, the monitoringsystem can take steps to mitigate the inefficiency, for example, byadjusting a setting of the appliance or by notifying a resident. In somecases, the monitoring system may be able to process thermal imagery of aproperty to improve the convenience and comfort of a resident of theproperty (e.g., by automatically adjusting a thermostat based on theresident's detected body temperature).

According to an innovative aspect of the subject matter described inthis application, a monitoring system is configured to monitor aproperty. The monitoring system includes a thermal camera that isconfigured to generate a thermal image of the property. The monitoringsystem includes a monitor control unit that is configured to receive,from the thermal camera, the thermal image; based on the thermal image,determine a temperature of a portion of the property depicted in thethermal image; determine that the temperature of the portion of theproperty depicted in the thermal image satisfies a temperaturethreshold; and based on determining that the temperature of the portionof the property depicted in the thermal image satisfies the temperaturethreshold, select and perform a monitoring system action.

These and other implementations can each optionally include one or moreof the following features. The monitoring system includes a sensor thatis configured to generate sensor data that reflects an attribute of theproperty. The action of selecting the monitoring system action is basedon the sensor data. The monitor control unit is configured to determinean arming status of the monitoring system. The action of selecting themonitoring system action is based on the arming status of the monitoringsystem. The monitoring system includes a thermostat that is configuredto generate temperature data that reflects an ambient temperature insidethe property. The temperature of the portion of the property depicted inthe thermal image is based on the temperature data. The monitoringsystem action is an action to deactivate an electric device that islocated at the property. The action to deactivate the electric deviceincludes identifying a circuit breaker powering the electric device; andtripping the circuit breaker powering the electric device.

The monitoring system action is an action to activate an electric devicethat is located at the property and that, upon activation, mitigates acondition depicted in the thermal image. The monitoring system action isan action that generates and transmits a notification to a resident ofthe property indicating the condition depicted in the thermal image. Thethermal camera is configured to generate location data that indicates alocation of the thermal camera within the property. The monitor controlunit is configured to receive, from the thermal camera, the locationdata; and determine the temperature threshold based on the locationdata. The monitor control unit is configured to determine a temperatureof a portion of the property depicted in the thermal image bydetermining the temperature of portion of the property that includes aresident of the property. The monitoring system action comprisesadjusting a setting of a thermostat of the property.

Other implementations of this aspect include corresponding systems,apparatus, and computer programs recorded on computer storage devices,each configured to perform the operations of the methods.

The details of one or more implementations of the subject matterdescribed in this specification are set forth in the accompanyingdrawings and the description below. Other features, aspects, andadvantages of the subject matter will become apparent from thedescription and the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an example of a system for propertycontrol and configuration based on thermal imaging.

FIG. 2 is a diagram illustrating an example of a thermal image.

FIGS. 3A, 3B, 3C, and 3D are diagrams illustrating examples of propertycontrol and configuration based on thermal imaging

FIG. 4 is a flow chart illustrating an example of a method for propertycontrol and configuration based on thermal imaging.

FIG. 5 is a diagram illustrating an example of a property monitoringsystem.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

FIG. 1 is a diagram illustrating an example of a system 100 for propertycontrol and configuration based on thermal imaging. The system 100includes a property 102 monitored by a property monitoring system. Themonitoring system includes one or more thermal imaging sensors, e.g.,the IR camera 130, which acquire thermal image data 133 from one or moreareas of the property 102. The monitoring system also includes amonitoring server 160, which analyzes the thermal image data, othersensor data, and/or additional monitoring system data to determine oneor more control and/or configuration actions related to the property102. FIG. 1 includes stages (A) through (E), which represent a flow ofdata.

The property 102 can be a home, another residence, a place of business,a public space, or another facility that is monitored by a propertymonitoring system. The monitoring system includes one or more sensors120 located at the property 102 that collect sensor data related to theproperty 102. For example, the sensors 120 can include motion detectorsthat sense movement in an area of the property 120. The sensors 120 canalso include appliance sensors, door or window lock sensors, utility orresource usage sensors, microphones, temperature or humidity sensors,light detectors, or other sensors.

The sensors 120 can also include one or more visible light cameras thatare in various locations of the property 102. The visible light cameracan capture image or video data of the physical surroundings detectablewithin the camera's field of view. In some implementations, the visiblelight camera may be paired with one or more motion sensors, wheredetection of movement by the motion sensor triggers the visible camerato capture image or video data. In some implementations, the visiblelight camera can include a microphone for capturing audio data detectedin the vicinity of the visible light camera. Other possible sensors aredescribed below in FIG. 5.

The monitoring system also includes one or more thermal imaging sensors,e.g., the IR cameras 130. The one or more IR cameras 130 acquire thermalimage data 133, e.g., based on detecting IR radiation in a particulararea of the property 102. The thermal image data 133 can be, forexample, IR images indicating the intensity or power of IR radiationreceived by the sensor array of the IR cameras 130. The IR cameras 130can include sensors that detect and respond to IR energy in one or moreIR spectral bands, including near-infrared (NIR) wavelengths (0.8 micronto 1.7 micron), short-wave infrared (SWIR) wavelengths (1 micron to 2.5micron), mid-wavelength infrared (MWIR) wavelengths (2 micron to 5micron) and long-wave infrared (LWIR) wavelengths (8 micron to 14micron).

The captured thermal image data 133 can include two-dimensional (2D)thermal image frames, as well as thermal image videos (e.g., a series of2D thermal image frames). In some implementations, the IR cameras 130are configured to acquire thermal images of one or more areas of theproperty 102 at periodic time intervals according to a capture interval135. For example, an IR camera 130 can be configured to acquire athermal image of the area once per minute (i.e., a capture interval 135of 1 minute), or once every 10 minutes (i.e., a capture interval 135 of10 minutes). The capture interval 135 may be set by default, or may beconfigurable by a user, e.g., by a resident 108 of the property 102. Insome implementations, the IR camera 130 can be configured to acquire athermal image when motion is detected within view of the IR camera 130(e.g., when movement is detected by a motion sensor or by analysis ofthe thermal image data 133).

The thermal image data 133 can be processed to estimate the relativeand/or absolute temperature of various objects imaged by the data 133.For example, for thermal image data 133 based on detection of radiatedIR energy, the temperature of an object can be estimated by processingthe pixels of the thermal image 133 according to a modifiedStefan-Boltzmann equation that accounts for transmissive IR lossesthrough the atmosphere:

$T_{obj} = \sqrt[4]{\frac{W - ( {1 - {ɛ_{obj}*\tau_{atm}*\sigma*( T_{refl} )^{4}} - {( {1 - \tau_{atm}} )*\sigma*( T_{atm} )^{4}}} }{ɛ_{obj}*\tau_{atm}*\sigma}}$

where, T_(obj) is the object temperature, W is the radiative power perunit area received by the camera at the wavelengths of interest, ε_(obj)is the object's emissivity at the wavelengths of interest, T_(atm) isthe atmospheric transmissivity at the wavelengths of interest, σ is theStefan-Boltzmann constant, T_(refl) is the reflected temperature, andT_(atm) is the atmospheric temperature.

The IR cameras 130 and the sensors 120 communicate with a control unit110 that is located at the property 102. The control unit 110 can be,for example, a computer system or other electronic device configured tocommunicate with the IR cameras 130 and sensors 120. The control unit110 can also perform various management tasks and functions for themonitoring system. In some implementations, the resident 108 of theproperty, or another user, can communicate with the control unit 110(e.g., input data, view settings, or adjust parameters) through aphysical connection, such as a touch screen or keypad, through a voiceinterface, and/or over a network connection.

The IR cameras 130 and the sensors 120 may communicate with the controlunit 110 through a network 105. The network 105 can be any communicationinfrastructure that supports the electronic exchange of data between thecontrol unit 110 and the one or more cameras 130 and sensors 120. Forexample, the network 105 may include a local area network (LAN). Thenetwork 105 may be any one or combination of wireless or wired networksand may include any one or more of Ethernet, Bluetooth, Bluetooth LE,Z-wave, Zigbee, or Wi-Fi technologies.

The one or more IR cameras 130 send thermal image data 133 to thecontrol unit 110 through the network 105. Similarly, the sensors 120send various sensor data to the control unit 110. For example, thesensors 120 can send motion sensing data from one or more motiondetectors, status data from one or more door or lock sensors (e.g., dooropen, lock secured), or light level data from one or more lightdetectors. The sensors 120 can also send image data from one or morevisible light cameras (e.g., visible light images and/or video) to thecontrol unit 110.

The control unit 110 can also communicate with one or more propertyautomation controls 122, possibly through the network 105. The propertyautomation controls 122 connect to one or more devices of the property102 and enable control of various property actions. For example, theproperty automation controls 122 can adjust a setting on a thermostat,disable one or more appliances, adjust a setting on an appliance, securea door lock, open a garage door, or control other devices of theproperty 102.

In stage (A), the one or more IR cameras 130 acquire thermal image data133 of an area of the property 102. In the example of FIG. 1, the IRcamera 130 is in a room of the property 102 and directed toward an areathat includes an electric iron 117 that has been left powered on (andthus hot) after the resident has left the property 102. The IR camera130 is configured to acquire thermal image data 133 of the area of theproperty 102 according to the capture interval 135, which, in FIG. 1,indicates that the camera 130 acquires a thermal image of the area onceevery 10 minutes. In the example of FIG. 1, the IR camera 130 acquiresthermal image data 133, which indicates that the surface of the iron 117is significantly hotter than the objects in the surrounding area. The IRcamera 130 then provides the thermal image data 133 to the control unit110 through the network 105.

Concurrently, other sensors 120 of the monitoring system may collectsensor data related to the same area of the property 102, or related toanother area of the property 102, and send the sensor data to thecontrol unit 110. For example, a motion detector may collect dataindicating that no movement is detected in the room near the electriciron 117, a visible light camera may indicate that there is no occupantin the room, and a light sensor may collect data indicating a low lightlevel in the room (e.g., the lights in the room are turned off).

The control unit 110 receives the image data 133 from the IR camera 130and the sensor data from the sensors 120. The control unit 110 may alsoreceive or generate other monitoring system data. For example, thecontrol unit 110 may receive data indicating a monitoring system statusor condition set by the resident 108 (e.g., “armed away,” “armed stay,”“unarmed,” etc.), and/or time and date information. The monitoringsystem data can also include information related to other devicesconnected to the monitoring system, for example, a status of a connecteddevice (e.g., whether particular lights are turned on, doors are locked,etc.) or a device setting (e.g., a thermostat setting, an appliancesetting, etc.). In some implementations, the monitoring system data caninclude data related to a mobile device 140 of the resident 108, wherethe mobile device 140 can be, for instance, a smart phone, cellularphone, tablet computer, laptop computer, a smart watch, or anothermobile computing device associated with the resident 108 or with anotherauthorized user of the monitoring system. In some examples, themonitoring system data can indicate whether the mobile device 140 ispresent at the property 102 (e.g., based on whether the device 140 isconnected to a Wi-Fi network at the property 102 or whether a GPS signalindicates that the device 140 is at the property 102).

In stage (B), the control unit 110 sends various data 111 to a remotemonitoring server 160, where the data 111 can include the thermal imagedata 133, sensor data from the sensors 120, and/or other monitoringsystem data. The monitoring server 160 may be, for example, one or morecomputer systems, server systems, or other computing devices that arelocated remotely from the property 102 and that are configured toprocess information related to the monitoring system at the property102. In some implementations, the monitoring server 160 is a cloudcomputing platform.

The control unit 110 communicates with the monitoring server 160 via along-range data link. The long-range data link can include anycombination of wired and wireless data networks. For example, thecontrol unit 110 can exchange information with the monitoring server 160through a wide-area-network (WAN), a broadband Internet connection, acellular telephony network, a wireless data network, a cable connection,a digital subscriber line (DSL), a satellite connection, or otherelectronic means for data transmission. The control unit 110 and themonitoring server 160 may exchange information using any one or more ofvarious communication synchronous or asynchronous protocols, includingthe 802.11 family of protocols, TCP/IP, GSM, 3G, 4G, 5G, LTE, CDMA-baseddata exchange or other techniques. In some implementations, thelong-range data link between the control unit 110 and the monitoringserver 160 is a secure data link (e.g., a virtual private network) suchthat the data exchanged between the unit 110 and the server 160 isencoded to protect against interception by an adverse third party.

In some implementations, various other monitoring system componentslocated at the property 102 communicate directly with the monitoringserver 160 (e.g., sending data directly to the monitoring server 160rather than sending data to the server 160 via the control unit 110).For example, one or more of the IR cameras 130, the sensors 120, theautomation controls 122, or other devices at the property 102 canprovide some or all of the data 111 to the monitoring server 160, e.g.,through an Internet connection.

In some implementations, the control unit 110 processes some or all ofthe data 111 before sending the data 111 to the monitoring server 160.For example, the control unit 110 may compress or encode the thermalimage data 133 to reduce the bandwidth required to support datatransmission. The control unit 110 can also aggregate, filter,transform, or otherwise process some or all of the data 111.

In the example of FIG. 1, the control unit 110 sends to the server 160data 111 that includes the thermal image data 133 indicating a hotelectric iron, sensor data from a motion detector indicating that nomovement is detected in the room, and monitoring system data indicatingthat the system status has been set to “armed away,” indicating that theresident 108 is likely not present at the property 102.

In stage (D), the monitoring server 160 analyzes the data 111 receivedfrom the control unit 110. The server 160 may analyze thermal image dataincluded in data 111 to (i) identify one or more objects in a thermalimage and (ii) to determine the absolute or relative temperature of theone or more objects. In some implementations, the server 160 can processa thermal image to generate a two-dimensional temperature map of some orall of the area of the property 102 captured in the thermal image. Theserver 160 can then determine the temperature of a particular object inthe thermal image data by determining the temperature of the pixels ofthe temperature map corresponding to the particular object.

The server 160 can analyze the data 111 to identify one or more objectsin the thermal image data by any of various image processing techniques.For example, the server 160 can apply feature extraction methods,histogram of oriented gradient techniques, scale-invariant featuretransforms, or other object-detection methods to the thermal image datato identify an object in the image. In some implementations, the server160 uses one or more deep-learning or machine learning approaches todetect and/or classify an object, such as neural networks, supportvector machines, decision trees, K-nearest neighbors or random forestmodels. In some implementations, the server 160 may perform facialrecognition or other processing to identify a particular resident 108depicted in the image data.

In some implementations, the server 160 may perform image processing toidentify an object of interest within a thermal image prior todetermining the temperature of the object. In these examples, ratherthan determining the temperature associated with every pixel in thethermal image data, the server 160 may only determine the temperature ofa subset of pixels, e.g., the subset of pixels associated with theobject of interest, and so reduce the computing resources required toprocess the thermal image data. In some implementations, the server 160may generate a 2D temperature map for an entire thermal image, then usethe temperature data to identify regions of the image that will beprocessed for object identification. In these examples, the server 160may be able to limit the thermal image data for which it performs objectdetection to those areas of an image that exhibit an anomalous (e.g.,unexpectedly high or low) temperature.

In some implementations, the server 160 applies other advanced dataprocessing techniques when analyzing a thermal image. For example, theserver 160 may apply wavelength-dependent processing techniques,temperature-dependent corrections, or additional techniques thatcompensate for known or detected heat sources (e.g., a stove) ortemperature variations (e.g., at a window) within the area of thethermal image. The server 160 may also apply processing related to theparticular IR camera 130 that acquired the thermal image (e.g.,processing to account for the sensitivity of the IR sensor array of thecamera 130). In some implementations, the server 160 may store or accessdata used to estimate an object's temperature from the received thermalimage data. For example, the server 160 may store or access a table ofobject emissivities at a plurality of IR wavelengths. In the example ofFIG. 1, the server 160 analyzes the thermal image data included in data111 to determine that the imaged area of the property 102 includes theelectric iron 117, which has a surface temperature of approximately 200degrees Fahrenheit, while the surrounding objects in the area have atemperature of approximately 68 degrees Fahrenheit.

The monitoring server 160 can also analyze sensor and/or monitoring dataincluded in the data 111 provided by the control unit 110 to determineone or more conditions of the property 102. For example, the server 160can analyze date and time information, motion detector data, or systemstatus information to determine an occupancy of the property 102 or anoccupancy of a particular area (e.g., a particular room) of the property102. In the example of FIG. 1, the server 160 analyzes the sensor dataindicating that no movement is detected on the property 102 andmonitoring system data indicating that the system status is “armed away”to determine that the property 102 is likely unoccupied.

In stage (D), in response to analyzing the data 111, the monitoringserver 160 determines one or more system actions 165. For example, theserver 160 can include a rules engine, which determines the systemactions 165 based on one or more rules 161. The rules 161 can be defaultrules, set in advance by a system administrator. The rules 161 can alsobe custom rules, set or modified by the resident 108 or anotherauthorized user of the monitoring system. The rules 161 may be general,such that they are applied to more than one property, or they may bespecific to the particular property 102. In some implementations, therules 161 can be customized according to the particular area of theproperty 102 (e.g., different rules 161 for different rooms of theproperty 102), the time of day, or other factors. In someimplementations, the rules 161 can be adjusted based on the analyzeddata 111.

In some implementations, the rules 161 include one or more thresholds162, where a threshold 162 can relate to the detected temperature of aparticular object or object type. The rules 161 can indicate that, undercertain circumstances, if the estimated temperature of an object imagedin an area of the property 102 is determined to satisfy the threshold162, the server 160 should perform one or more actions 165. In theexample of FIG. 1, the rules 161 indicate that if an electric iron isdetected to have a temperature greater than a threshold 162 of 80degrees Fahrenheit when the property 102 is likely unoccupied, theserver 160 should send a notification to the mobile device 140 of theresident 108 and disable power delivery to the electric outlet connectedto the electric iron.

In some implementations, the monitoring server 160 may adjust one ormore of the thresholds 162 based on the analyzed data 111 (e.g., thermalimage data, sensor data, and/or other monitoring system data). Forexample, if sensor data from multiple motion sensors 120 indicates thatthere is no movement in any room of the property 102, the property 102may be unoccupied. As a result, the server 160 may set the temperaturethreshold 162 for the electric iron 117 to 80 degrees, as indicatedabove, so that an action 165 is performed if the electric iron 117 isdetected to be warm while the property 102 is unoccupied. However, ifthe motion sensor data indicates that there is movement in the room ofthe property 102 in which the electric iron 117 is located, the resident108 may be using the electric iron 117. As a result, the server 160 mayadjust the temperature threshold 162 for the electric iron to 500degrees, so that an action 165 is performed only if the temperature ofthe electric iron 117 is detected to be above a safe operatingtemperature while the resident 108 is using it.

In some implementations, the rules 161 may indicate that the temperatureof an object must satisfy the threshold 162 for a particular time periodin order for the monitoring server 160 to perform the one or moreactions 165. For example, the rules 161 may indicate that the monitoringserver 160 should send a notification to the mobile device 140 if theelectric iron is detected to have a temperature greater than 80 degreesFahrenheit for more than 10 minutes.

In some implementations, the monitoring server 160 may adjust the timeperiod associated with a particular threshold 162 based on analyzing thedata. For example, the server 160 may set the time period associatedwith the threshold 162 to one value when the monitoring system status isset to “armed stay” (e.g., indicating that the resident 108 is presentat the property 102) and to a second, different value when themonitoring system status is set to “armed away.” In the example of FIG.1, the server 160 may set the time period associated with thetemperature threshold 162 for the electric iron 117 to a relativelyshort time period (e.g., 10 minutes) when the monitoring system statusis “armed away,” and a longer time period (e.g., 60 minutes) when themonitoring system status is “armed stay,” since the resident 108 may beusing the electric iron 117 if he is at the property 102.

Generally, the resident 108 can customize the one or more rules 161 andthresholds 162 according to his preferences. In some implementations,the resident 108 can set the one or more rules 161 and/or thresholds 162through a software application executing on his mobile device 140,through a graphical interface provided by a browser or application on acomputing device, and/or through interacting with a physical interfaceof the control unit 110 of the property monitoring system.

The server 160 can determine any of various actions 165 in response toanalyzing the data 111. For example, the server 160 may determineactions 165 that include sending a notification to a mobile device 140of the resident 108, sending an instruction to the automation controls122 to adjust a setting of a connected device or appliance, sending acommand to a sensor 120 to collect sensor data, sounding an alarm of theproperty 102, or sending an alert to a third-party, such as securitypersonnel or emergency services.

In some implementations, the actions 165 may include sending a messageto the mobile device 140 of the resident 108 and requesting a responsefrom the resident 108. For example, the monitoring server 160 can send amessage to the mobile device 140 requesting permission to adjust theconfiguration of an automation control 122. The server 160 can waituntil it receives an affirmative response from the resident 108 (e.g.,an affirmative response sent from the mobile device 140 of the resident108) before adjusting the configuration of the particular automationcontrol 122.

In stage (E), the server 160 performs the system actions 165. Forexample, the server 160 can perform the actions 165 by sending a commandto a device of the monitoring system by sending an instruction to thecontrol unit 110 over the long-range data link. In some implementations,the server 160 can send a notification and/or alert to the mobile device140 of the resident 108. The server 160 can communicate with the mobiledevice 140 through a cellular telephony or wireless data network,through a WAN or LAN, through Wi-Fi, or through another wired orwireless communication link. The notification can be, for example, anSMS text message, an e-mail, a message displayed by an applicationexecuting on the mobile device 140, or another notification or messagesent to the device 140.

In the example of FIG. 1, the server 160 performs the actions 165 bysending the notification 141 to the mobile device 140 of the resident108 and by sending an instruction to the control unit 110 to disablepower delivery to the electric outlet to which the iron 117 isconnected.

In some implementations, the server 160 includes one or more machinelearning modules to analyze the data 111, generate the rules 161,generate the thresholds 162, and/or determine the actions 165. Forexample, the server 160 can include one or more neural networks, linearor logistic regression models, decision trees, support vector machines,Bayesian techniques, nearest-neighbor or clustering techniques, or othermachine learning approaches. The machine learning modules of server 160may support supervised and/or unsupervised learning.

In some implementations, the machine learning modules may supportsupervised learning using labeled training data. The labeled trainingdata can, for instance, be thermal image data 133 acquired by the IRcamera 130, where the image data 133 has been labeled by the resident108, by another user, or by a computer-implemented algorithm (e.g.,indicating that the thermal image data 133 corresponds to an appliancein its usual or default state).

In some implementations, the server 160 analyzes multiple thermal imagesto determine a baseline temperature of one or more objects or regions ofthe property. For example, the server 160 can analyze multiple thermalimages of an area of the property 102 that have been acquired over aparticular time span (e.g., multiple thermal images acquired over thecourse of an hour or over the course of a day) to determine a usualtemperature for the area of the property 102. The server 160 can thenuse the determined usual temperatures to determine the thresholds 162.

In some implementations, the server 160 analyzes multiple thermal imagesto determine the system actions 165. For example, in someimplementations, the thermal data 133 may include a timestamp and theserver 160 may analyze a series of thermal images of the same area ofthe property 102 acquired at different times to determine thetemperature of an object or an area of the property 102 as a function oftime. The server 160 can then determine the one or more actions 165based on the temperature as a function of time.

In some implementations, the server 160 may adjust the capture interval135 based on the analyzed thermal image data, sensor data, or othermonitoring system data. For example, the capture interval 135 mayinitially be set to 30 minutes, where the IR camera 130 acquires thermalimage data 133 once every 30 minutes. The server 160 may analyze thethermal image data to determine that the electric iron 117 is hotterthan the threshold temperature 162. Based on the determination, theserver 160 may reduce the capture interval to five minutes, such thatthe IR camera 130 now acquires thermal image data 133 once every fiveminutes until the server 160 determines that the electric iron 117 hasreturned to its usual temperature less than the threshold temperature162. In another example, the server 160 may adjust the capture interval135 to one value when the monitoring system status is set to “armedstay,” (e.g., to a shorter interval to capture thermal images of theproperty 102 more frequently while residents are present), and adjustthe capture interval 135 to a second value when the monitoring systemstatus is “armed away” (e.g., to a longer interval to capture thermalimages of the property 102 less frequently when the property 102 isunoccupied).

Though the system and method of FIG. 1 is described in the context of anelectric iron 117 that has been left powered, it could be similarlyapplied to other heat-generating appliances or devices, such as anelectric or gas stove, a furnace, a clothes dryer, a space heater, acomputer system, a fan, or another device that generates heat duringoperation.

Though described above as being performed by a particular component ofsystem 100 (e.g., the control unit 110 or the monitoring server 160),any of the various control, processing, and analysis operations can beperformed by either the control unit 110, the monitoring server 160, oranother computer system of the system 100. For example, the control unit110, the monitoring server 160, or another computer system can analyzethe thermal image data, sensor data, and monitoring system data 111 todetermine the actions 165. Similarly, the control unit 110, themonitoring server 160, or another computer system can control thevarious sensors 120, the IR camera 130, and/or the property automationcontrols 122 to collect data or control device operation.

FIG. 2 is a diagram illustrating an example of a thermal image 200, suchas may be included in the thermal image data 133 recorded by the IRcamera 130 of system 100. The thermal image 200 includes atwo-dimensional array of pixels, where each pixel indicates an amount ofradiative IR power received by the IR camera's sensor array from aparticular area of the scene. The thermal image 200 may be singlechannel (e.g., grayscale) or may include multiple channels, where themultiple channels correspond to radiative IR power received in differentIR spectral bands. In some implementations, a camera may include both IRand visible light sensors, such that the thermal image 200 can beaccompanied by a corresponding visible light image of the same area ofthe property.

In the example thermal image 200, the varying grayscale corresponds todifferent levels of IR power received by the IR camera's sensor array.Based on known or estimated emissivity values of the objects within theimage, the pixel data of the thermal image 200 can be processed togenerate a two-dimensional temperature map of all or part of thecaptured image.

Processing of one or more thermal images 200 can be performed by acomputer system of the IR camera 130, the control unit 110, themonitoring server 160, or another computer system that is part of theproperty monitoring system. The thermal images 200 can be processedaccording to any of various techniques to determine the relative orabsolute temperature of one or more regions of the property 102. Forexample, the values of the pixels of an image can be mapped to any of anabsolute temperature scale, a relative temperature scale, or a colorscale to generate a heat map. If the thermal image 200 is accompanied bya corresponding visible light image, some or all of the image processingperformed by the system (e.g., object detection and classification) maybe performed on the visible light image of the area.

A property monitoring system can use thermal imaging data acquired by athermal imaging sensor of the property in various ways to enhance thesafety and efficiency of a property or to improve the comfort andconvenience of one or more residents of the property. FIGS. 3A, 3B, 3Cand 3D are diagrams illustrating examples 300, 320, 340, and 360,respectively, of property control and configuration based on thermalimaging.

Example 300 of FIG. 3A includes an area 302 in a kitchen viewed by an IRcamera that is part of a property monitoring system. The area 302includes a refrigerator 307 whose door has been left slightly ajar afterthe resident has left the kitchen. The IR camera located in the kitchencaptures thermal image data 304 of the area 302, with the thermal imagedata 304 schematically represented in the center panel of FIG. 3A. Thethermal image data 304 includes a region 309, corresponding to theexposed, open region of the refrigerator 307, from which cold air isescaping. The region 309 appears darker than the surrounding area,indicating that the sensor array of the IR camera received lessradiative IR power from the region 309 of the area 302 due to the coldertemperature of the region 309.

A computer system of the property monitoring system, for example, themonitoring server 160 or the control unit 110 of system 100, processesthe thermal image data 304 to determine (i) that the region 309corresponds to a portion of the refrigerator 307, (ii) that the region309 is at a temperature of approximately 30 degrees Fahrenheit, which iscolder than its surroundings, and (iii) that the region 309 is detectedto be colder than a threshold temperature of 40 degrees Fahrenheit,which is the usual temperature of region 309 when the door of therefrigerator 307 is fully closed. Based on these determinations, thecomputer system of the monitoring system determines that the door of therefrigerator 307 is open.

In addition to the thermal image data 304, the computer system receivessensor data from a motion detector indicating that no movement isdetected in the kitchen. Based on the sensor data, the system sets thecapture interval for the IR camera in the kitchen to five minutes. Afterfive minutes, the IR camera captures a second thermal image thatindicates that the refrigerator door is still open. Based on thereceived thermal image data and motion sensor data, the systemdetermines that the door of the refrigerator 307 has been open for atleast five minutes and the resident is not in the kitchen.

As shown in the results panel 306, in response to determining that thedoor of the refrigerator 307 has been left open and the resident is notin the kitchen, the monitoring system performs the actions 365A, whichinclude sending a message to an authorized mobile device (e.g., a smartphone or tablet computing device) of the resident notifying him that therefrigerator door is ajar. By sending a notification to the resident'smobile device, the system alerts the resident to the open refrigeratordoor in a timely manner, enabling the resident to take corrective action(e.g., to close the door) before significant energy is wasted or foodspoiled.

FIG. 3B illustrates another example 320 of property control andconfiguration based on thermal imaging. Example 320 also includes anarea 322 in a kitchen viewed by an IR camera that is part of a propertymonitoring system. The area 322 includes an electric toaster 327 that isplugged into an outlet 323, which provides electric power to the toaster327. In the area 322, the connection between the plug of the toaster 327and the outlet 323 is compromised (e.g., the toaster plug is looselyinserted into the outlet receptacle), resulting in heating of the outletarea surrounding the connection and presenting a fire hazard.

The IR camera located in the kitchen captures thermal image data 324 ofthe area 322, with the thermal image data 324 schematically representedin the center panel of FIG. 3B. The thermal image data 324 includes aregion 329, corresponding to the area of the outlet surrounding theconnection between the toaster plug and the outlet receptacle. Theregion 329 appears brighter than the surrounding area, indicating thatthe sensor array of the IR camera received more radiative IR power fromthat region of the area 322 because it is hotter than the surroundingarea.

A computer system of the property monitoring system processes thethermal image data 324 to determine (i) that the region 329 correspondsto an area of the outlet 323 surrounding the location where the toaster327 connects to the electrical outlet 323 and (ii) that the region 329is at a temperature greater than (i.e., is hotter than) a thresholdtemperature that represents the safe operating temperature forelectrical connections at the property.

As depicted in the results panel 236, based on determining that thetemperature of the area near the electrical connection is greater thanthe safe operating temperature, the monitoring system performs theactions 365B, which include interrupting power delivery to the toaster327 and sending a message to the authorized mobile device of theresident notifying him that the outlet 323 was detected to be overheatedand that the toaster 327 has been powered-off to mitigate the risk offire.

FIG. 3C illustrates yet another example 340 of property control andconfiguration based on thermal imaging. Example 340 includes an area 342in a living room of a home. The area 342 is viewed by an IR camera thatis part of the home's property monitoring system. The area 342 includesa window 347 on an exterior wall that is equipped with a smart windowshade 343. The smart window shade 343 is configured to communicate withthe home's property monitoring system, such that the system can controlthe position of the shade 343. For example, the monitoring system cansend an instruction to the window shade 343 to raise itself, allowinglight from outside of the window 343 to pour into the room.Alternatively, the monitoring system can send an instruction to thewindow shade 343 to lower itself, blocking light from outside of thewindow 343 from entering the room. In scene 342, the window shade 343 israised, allowing light 349 to enter the room through the window 343.

The IR camera located in the living room captures thermal image data 344of the area 322, with the thermal image data 344 schematicallyrepresented in the center panel of FIG. 3C. The thermal image data 344indicates that the area 351 of the wall surrounding illuminated by thelight 349 is brighter than the area of the wall that is not illuminated,indicating that the illuminated area 351 is hotter than the other areasof the wall.

A computer system of the property monitoring system processes thethermal image data 344 to determine that the wall area 351 has atemperature of 80 degrees Fahrenheit. The system also receivesmonitoring system data indicating that the monitoring system status is“armed stay” and that the thermostat of the property is currentlyconfigured to maintain a target ambient temperature of 75 degreesFahrenheit. Because the monitoring system data indicates that at leastone resident is at the property (e.g., the system status is “armedstay”) and the target ambient temperature is 75 degrees Fahrenheit, thesystem adjusts the threshold temperature for the wall area 351 to 75degrees Fahrenheit.

As shown in the results panel 346, based on determining that thetemperature of the wall area 351 is greater than the threshold, themonitoring system performs the actions 365C, which include sending aninstruction to the smart window shade 347 to lower itself, adjusting asetting of a thermostat of the property to increase air flow to theliving room to cool the room, and sending a message to the authorizedmobile device of the resident notifying him of the actions. Byautomatically lowering the window shade 347 and adjusting thethermostat, the property monitoring system can enhance the energyefficiency of the home and improve the comfort of the residents of theproperty.

FIG. 3D illustrates yet another example 360 of property control andconfiguration based on thermal imaging. Example 360 includes an area 362in a room of a home, which is viewed by an IR camera that is part of thehome's property monitoring system. In the area 362, the resident 368 hasjust returned to the home after engaging in strenuous exercise and, as aresult, the resident's body temperature is elevated compared to theresident's usual body temperature.

The IR camera located in the room captures thermal image data 364 of thearea 360, with the thermal image data 364 schematically represented inthe center panel of FIG. 3D. The thermal image data 364 indicates thatthe body temperature of the resident 368 is detected to be higher thanthe usual body temperature of the resident 368.

A computer system of the property monitoring system processes thethermal image data 364 to determine (i) that the resident 368 is presentin the area imaged by the thermal image data 364 and (ii) that thedetected temperature of the resident 368 is greater than a thresholdtemperature that corresponds to the resident's usual temperature. Asshown in the results panel 366, based on determining that the resident'stemperature is greater than the threshold, the monitoring systemperforms the actions 365D, which include adjusting a setting of athermostat (e.g., turning on a fan or lowering the thermostat set-pointtemperature to cool the room) and sending a message to the authorizedmobile device of the resident 368 notifying him that the thermostatsetting has been adjusted. By automatically detecting the bodytemperature of the resident 368 and adjusting the thermostataccordingly, the property monitoring system can enhance the comfort ofthe resident 368.

In some implementations, the processing performed by the computer systemincludes facial recognition or other processing to identify theparticular resident 368 depicted in the thermal image data 364. In thiscase, the system can customize its response for the identified resident368 by applying different threshold temperatures for differentidentified residents 368 (e.g., if one resident prefers the house to beat one temperature and another resident prefers the house to be at adifferent temperature).

The examples described in FIGS. 3A through 3D are only several of themany implementations of property control and configuration based onthermal imaging that are contemplated within the scope of thisdisclosure.

For instance, in some implementations, the monitoring system can usethermal image data to monitor a property for conditions that requiremaintenance or remediation. For example, a thermal camera of theproperty may acquire thermal image data from areas of the home nearvents that connect to the property's heating ventilation and airconditioning (HVAC) system. The monitoring system can analyze thethermal image data to detect any unusual or unexpected change intemperature near the vents (e.g., a vent that is unusually cold when theheat is activated) to identify a potential malfunction of the HVACsystem. The system can then adjust a setting of the HVAC system and/orsend a notification to the resident's mobile device indicating thepotential malfunction.

As another example, a thermal camera at a home may acquire thermal imagedata of a ceiling of the home at regular intervals (e.g., once per day).The monitoring system can process the thermal image data to determinewhether any region of the ceiling changes temperature relative to otherareas of the ceiling (e.g., whether a cold spot or a hot spot develops).If the temperature of a region of the ceiling changes relative to thetemperatures of other areas of the ceiling, the monitoring system candetermine there is a potential water leak above that region of theceiling. The system can then send a notification to the resident'smobile device indicating the potential leak.

In some implementations, the thermal camera may be located in a garageof the property. By analyzing thermal image data acquired in the garage,the monitoring system may be able to determine that a parked vehicle isin an unexpected condition, for instance, the vehicle's headlights havebeen left on (e.g., because the headlights are at an elevatedtemperature) or that the vehicle's engine has been left running (e.g.,because the car hood is at an elevated temperature). If the monitoringsystem determines that the parked vehicle has been in the unexpectedcondition for a predetermined time (e.g., for 30 minutes), it can send anotification the resident's mobile device alerting him to the unexpectedvehicle condition.

In some implementations, the thermal image data acquired by the thermalcamera includes thermal images of people and/or animals. In this case,much like the example of FIG. 3D, the monitoring system may process thethermal image data to determine the body temperature of the peopleand/or animals. Based on the body temperatures of the people and/oranimals, the system can adjust one or more HVAC or thermostat settings(e.g., a set-point temperature, a fan setting, etc.). For example, thesystem can automatically adjust the thermostat in response to a pet'sbody temperature, ensuring that the pet is comfortable in the home whenthe resident is away from the home. If more than one person or animal isdetected in the thermal image data, the monitoring system may processthe image data to determine the number of people and their bodytemperature distribution (e.g., the average body temperature, themaximum body temperature, and/or the minimum body temperature), and thenadjust the HVAC or thermostat setting in response to the temperaturedistribution. By adjusting the HVAC setting based on the analyzedthermal image data rather than simply on a thermostat-measured roomtemperature, the system can account for a particular condition of aperson (e.g., whether they have just come in from the cold) or an animal(e.g., whether a pet dog has a particularly thick coat) that impactstheir comfort but is not captured by the thermostat measurement. In someimplementations, the system may process the thermal images to identify aparticular person or animal (e.g., through facial recognition or objectrecognition) and set the threshold based on the identified person oranimal.

FIG. 4 is a flow chart illustrating an example of a method 400 forproperty control and configuration based on thermal imaging. Method 400can be performed by one or more computer systems, for example, themonitoring server 160 of system 100. In some implementations, some orall of the method can be performed by the control unit 110 of the system100, or by another computer system located at the monitored property.Briefly, method 400 includes receiving, from a thermal camera of aproperty, a thermal image (402); analyzing the thermal image (404);based on analyzing the thermal image, determining a temperature of anarea of the property (406); comparing the temperature of the area of theproperty to a threshold (408); based on comparing the temperature of thearea of the property to the threshold, determining that the temperatureof the area of the property satisfies the threshold (410); and, based ondetermining that the temperature of the area of the property satisfiesthe threshold, performing one or more actions (412).

In more detail, the monitoring server or another computer systemreceives, from a thermal camera of a property, a thermal image (402).The thermal camera can be, for example, an IR camera or another thermalimaging sensor. In some implementations, the monitoring server is remotefrom the property and a control unit located at the property sends thethermal image to the monitoring server over a long-range data link. Thethermal image can include data related to the radiated IR power receivedby an IR imaging sensor (e.g., from an IR camera) from various regionsof an area of the property within the field-of-view of the IR camera. Insome implementations, the server receives more than one thermal image ofthe area of the property, for example, a series of thermal images of thesame area of the property acquired at different times, or a thermalvideo of the scene of the property.

The server or other computer system analyzes the thermal image (404).For instance, the server can process the thermal image to identify theportion of the property captured by the image. In some examples, theserver may identify one or more objects, people, or animals captured bythe thermal image. In some examples, the server may perform facialrecognition or object classification to identify an object, person, oranimal captured by the thermal image. The server may also analyze thethermal image data to determine an absolute or relative temperature ofone or more objects, people, animals or areas of the property depictedin the thermal image. In some examples, the server may generate atemperature map based on the thermal image data, where the temperaturemap indicates an estimated temperature for each region of the image(e.g., on a pixel-by-pixel basis). The server can determine thetemperature of various regions of the image by applying any of variousprocessing techniques, such as processing based on the Stefan-Boltzmannrelation.

In some implementations, the server may analyze the thermal image alongwith other data provided by the monitoring system of the property. Forexample, the server may analyze thermal image data in conjunction withother sensor data collected by sensors located at the property, or withother monitoring system data, such as the monitoring system status, or apredicted location of a resident. In some implementations, the servermay include one or more machine learning modules that analyze thethermal image and/or other data (e.g., sensor data or monitoring systemdata).

Based on analyzing the thermal image, the server or other computersystem determines a temperature of an area of the property (406). Forexample, the server may determine the temperature of an object, aperson, or an animal identified within the thermal image of theproperty. The server can determine the temperature of an appliance or apart of an appliance. The server can determine the temperature of aportion of a wall, floor, or ceiling. The determined temperature of thearea of the property can be an absolute temperature (e.g., in degreesFahrenheit or Celsius) or a relative temperature (e.g., a temperature ofthe area relative to a reference temperature).

After determining a temperature of an area of the property, the serveror other computer system compares the determined temperature to athreshold (408). In some implementations, the threshold includes atemperature. The system can compare the determined temperature to anabsolute temperature threshold (e.g., compare whether the determinedtemperature is greater than 95 degrees Fahrenheit) or a relativetemperature threshold (e.g., compare whether the determined temperatureis at least five degrees Fahrenheit greater than the temperature ofanother area of the property). In some implementations, the thresholdalso includes an associated time period, such that the determinedtemperature of the area of the property must be above (or below) theindicated temperature for the indicated associated time period for thethreshold to be satisfied. In some examples, the threshold may be atemperature ratio or other criterion.

The threshold can be predetermined by the system (e.g., a defaultthreshold). In some implementations, the threshold can be generated oradjusted by the system based on analysis of the current and/orhistorical data received from the monitoring system (e.g., currentand/or historical thermal image data, sensor data, and other monitoringsystem data). For example, based on sensor data (e.g., motion sensingdata, visible image data, other sensor data) and/or monitoring systemdata (e.g., the system status, thermostat data), the server can adjust athreshold temperature, a time period associated with a thresholdtemperature, or another threshold parameter. In some implementations,the system may also use the analyzed data to adjust an operationalsetting of the system (e.g., a capture interval, a setting of thethermal camera).

In some implementations, the threshold can be adjusted by a user. Forexample, a resident of the property can set a value for a thresholdtemperature or time through an application on a personal mobile ornon-mobile computing device. The threshold can be stored by a memorysystem of the server and accessed by the server.

In some implementations, the threshold is included within one or morerules that the server applies to the analyzed thermal image data, wherethe one or more rules indicate one or more actions that the servershould perform if the threshold is satisfied.

Based on comparing the temperature of the area of the property to thethreshold, the server determines that the temperature of the area of theproperty satisfies the threshold (410). For example, the threshold maybe a maximum safe operating temperature and the server may determinethat an appliance has a temperature above the maximum safe operatingtemperature, satisfying the threshold. In some implementations, thethreshold may be a preferred range of temperatures and the serverdetermines that the detected temperature of the area of the propertysatisfies the threshold if the detected temperature is outside of thepreferred range of temperatures.

Based on determining that the temperature of the area of the propertysatisfies the threshold, the server performs one or more actions (412).The server can determine the one or more actions based on one or morerules accessed by the server. In some implementations, the one or morerules may be set by a user, for instance, the resident may customize therules using a software application on a computing device thatcommunicates with the monitoring system.

The server can perform any of various actions based on determining thatthe temperature of the property satisfies the threshold. For example,the server can send an instruction to a device of the property toperform an action, such as disabling or enabling the device, adjusting asetting of the device, or triggering an operation of the device. Theserver can activate or change the operating mode of a sensor of theproperty, or sound an alarm of the property.

The server can also send a message to a mobile device of a user orresident, for example, a text message, an e-mail message, a pushnotification, a message through a software application, or anotheralert. In some implementations, the server may send a message to themobile device of a user requesting permission to perform another actionof the monitoring system. For example, the server may request permissionto adjust a thermostat setting, or deactivate an electrical outlet. Insome cases, the server may send a message to a third-party, such assecurity or emergency-services personnel.

FIG. 5 is a diagram illustrating an example of a property monitoringsystem 500. The system 500 includes a network 505, a control unit 510,one or more user devices 540 and 550, a monitoring server 560, and acentral alarm station server 570. In some examples, the network 505facilitates communications between the control unit 510, the one or moreuser devices 540 and 550, the monitoring server 560, and the centralalarm station server 570.

The network 505 is configured to enable exchange of electroniccommunications between devices connected to the network 505. Forexample, the network 505 may be configured to enable exchange ofelectronic communications between the control unit 510, the one or moreuser devices 540 and 550, the monitoring server 560, and the centralalarm station server 570. The network 505 may include, for example, oneor more of the Internet, Wide Area Networks (WANs), Local Area Networks(LANs), analog or digital wired and wireless telephone networks (e.g., apublic switched telephone network (PSTN), Integrated Services DigitalNetwork (ISDN), a cellular network, and Digital Subscriber Line (DSL)),radio, television, cable, satellite, or any other delivery or tunnelingmechanism for carrying data. The network 505 may include multiplenetworks or subnetworks, each of which may include, for example, a wiredor wireless data pathway. The network 505 may include a circuit-switchednetwork, a packet-switched data network, or any other network able tocarry electronic communications (e.g., data or voice communications).For example, the network 505 may include networks based on the Internetprotocol (IP), asynchronous transfer mode (ATM), the PSTN,packet-switched networks based on IP, X.25, or Frame Relay, or othercomparable technologies and may support voice using, for example, VoIP,or other comparable protocols used for voice communications. The network505 may include one or more networks that include wireless data channelsand wireless voice channels. The network 505 may be a wireless network,a broadband network, or a combination of networks including a wirelessnetwork and a broadband network.

The control unit 510 includes a controller 512 and a network module 514.The controller 512 is configured to control a control unit monitoringsystem (e.g., a control unit system) that includes the control unit 510.In some examples, the controller 512 may include a processor or othercontrol circuitry configured to execute instructions of a program thatcontrols operation of a control unit system. In these examples, thecontroller 512 may be configured to receive input from sensors, flowmeters, or other devices included in the control unit system and controloperations of devices included in the household (e.g., speakers, lights,doors, etc.). For example, the controller 512 may be configured tocontrol operation of the network module 514 included in the control unit510.

The network module 514 is a communication device configured to exchangecommunications over the network 505. The network module 514 may be awireless communication module configured to exchange wirelesscommunications over the network 505. For example, the network module 514may be a wireless communication device configured to exchangecommunications over a wireless data channel and a wireless voicechannel. In this example, the network module 514 may transmit alarm dataover a wireless data channel and establish a two-way voice communicationsession over a wireless voice channel. The wireless communication devicemay include one or more of a LTE module, a GSM module, a radio modem,cellular transmission module, or any type of module configured toexchange communications in one of the following formats: LTE, GSM orGPRS, CDMA, EDGE or EGPRS, EV-DO or EVDO, UMTS, or IP.

The network module 514 also may be a wired communication moduleconfigured to exchange communications over the network 505 using a wiredconnection. For instance, the network module 514 may be a modem, anetwork interface card, or another type of network interface device. Thenetwork module 514 may be an Ethernet network card configured to enablethe control unit 510 to communicate over a local area network and/or theInternet. The network module 514 also may be a voice band modemconfigured to enable the alarm panel to communicate over the telephonelines of Plain Old Telephone Systems (POTS).

The control unit system that includes the control unit 510 includes oneor more sensors 520. For example, the monitoring system may includemultiple sensors 520. The sensors 520 may include a lock sensor, acontact sensor, a motion sensor, or any other type of sensor included ina control unit system. The sensors 520 also may include an environmentalsensor, such as a temperature sensor, a water sensor, a rain sensor, awind sensor, a light sensor, a smoke detector, a carbon monoxidedetector, an air quality sensor, etc. The sensors 520 further mayinclude a health monitoring sensor, such as a prescription bottle sensorthat monitors taking of prescriptions, a blood pressure sensor, a bloodsugar sensor, a bed mat configured to sense presence of liquid (e.g.,bodily fluids) on the bed mat, etc. In some examples, the healthmonitoring sensor can be a wearable sensor that attaches to a user inthe home. The health monitoring sensor can collect various health data,including pulse, heart-rate, respiration rate, sugar or glucose level,bodily temperature, or motion data.

The sensors 520 can also include a radio-frequency identification (RFID)sensor that identifies a particular article that includes a pre-assignedRFID tag.

The system 500 also includes one or more thermal cameras 530 thatcommunicate with the control unit 510. The thermal camera 530 may be anIR camera or other type of thermal sensing device configured to capturethermal images of a scene. For instance, the thermal camera 530 may beconfigured to capture thermal images of an area within a building orhome monitored by the control unit 510. The thermal camera 530 may beconfigured to capture single, static thermal images of the area and alsovideo thermal images of the area in which multiple thermal images of thearea are captured at a relatively high frequency (e.g., thirty imagesper second). The thermal camera 530 may be controlled based on commandsreceived from the control unit 510. In some implementations, the thermalcamera 530 can be an IR camera that captures thermal images by sensingradiated power in one or more IR spectral bands, including NIR, SWIR,MWIR, and/or LWIR spectral bands.

The thermal camera 530 may be triggered by several different types oftechniques. For instance, a Passive Infra-Red (PIR) motion sensor may bebuilt into the thermal camera 530 and used to trigger the thermal camera530 to capture one or more thermal images when motion is detected. Thethermal camera 530 also may include a microwave motion sensor built intothe camera and used to trigger the thermal camera 530 to capture one ormore thermal images when motion is detected. The thermal camera 530 mayhave a “normally open” or “normally closed” digital input that cantrigger capture of one or more thermal images when external sensors(e.g., the sensors 520, PIR, door/window, etc.) detect motion or otherevents. In some implementations, the thermal camera 530 receives acommand to capture an image when external devices detect motion oranother potential alarm event. The thermal camera 530 may receive thecommand from the controller 512 or directly from one of the sensors 520.

In some examples, the thermal camera 530 triggers integrated or externalilluminators (e.g., Infra-Red or other lights controlled by the propertyautomation controls 522, etc.) to improve image quality. An integratedor separate light sensor may be used to determine if illumination isdesired and may result in increased image quality.

The thermal camera 530 may be programmed with any combination oftime/day schedules, monitoring system status (e.g., “armed stay,” “armedaway,” “unarmed”), or other variables to determine whether images shouldbe captured or not when triggers occur. The thermal camera 530 may entera low-power mode when not capturing images. In this case, the thermalcamera 530 may wake periodically to check for inbound messages from thecontroller 512. The thermal camera 530 may be powered by internal,replaceable batteries if located remotely from the control unit 510. Thethermal camera 530 may employ a small solar cell to recharge the batterywhen light is available. Alternatively, the thermal camera 530 may bepowered by the controller's 512 power supply if the thermal camera 530is co-located with the controller 512.

In some implementations, the thermal camera 530 communicates directlywith the monitoring server 560 over the Internet. In theseimplementations, thermal image data captured by the thermal camera 530does not pass through the control unit 510 and the thermal camera 530receives commands related to operation from the monitoring server 560.

In some implementations, the system 500 includes one or more visiblelight cameras, which can operate similarly to the thermal camera 530,but detect light energy in the visible wavelength spectral bands. Theone or more visible light cameras can perform various operations andfunctions within the property monitoring system 500. For example, thevisible light cameras can capture images of one or more areas of theproperty, which the cameras, the control unit 110, and/or anothercomputer system of the monitoring system 500 can process and analyze.

The system 500 also includes one or more property automation controls522 that communicate with the control unit 110 to perform monitoring.The property automation controls 522 are connected to one or moredevices connected to the system 500 and enable automation of actions atthe property. For instance, the property automation controls 522 may beconnected to one or more lighting systems and may be configured tocontrol operation of the one or more lighting systems. Also, theproperty automation controls 522 may be connected to one or moreelectronic locks at the property and may be configured to controloperation of the one or more electronic locks (e.g., control Z-Wavelocks using wireless communications in the Z-Wave protocol). Further,the property automation controls 522 may be connected to one or moreappliances at the property and may be configured to control operation ofthe one or more appliances. The property automation controls 522 mayinclude multiple modules that are each specific to the type of devicebeing controlled in an automated manner. The property automationcontrols 522 may control the one or more devices based on commandsreceived from the control unit 510. For instance, the propertyautomation controls 522 may interrupt power delivery to a particularoutlet of the property or induce movement of a smart window shade of theproperty.

The system 500 also includes thermostat 534 to perform dynamicenvironmental control at the property. The thermostat 534 is configuredto monitor temperature and/or energy consumption of an HVAC systemassociated with the thermostat 534, and is further configured to providecontrol of environmental (e.g., temperature) settings. In someimplementations, the thermostat 534 can additionally or alternativelyreceive data relating to activity at the property and/or environmentaldata at the home, e.g., at various locations indoors and outdoors at theproperty. The thermostat 534 can directly measure energy consumption ofthe HVAC system associated with the thermostat, or can estimate energyconsumption of the HVAC system associated with the thermostat 534, forexample, based on detected usage of one or more components of the HVACsystem associated with the thermostat 534. The thermostat 534 cancommunicate temperature and/or energy monitoring information to or fromthe control unit 510 and can control the environmental (e.g.,temperature) settings based on commands received from the control unit510.

In some implementations, the thermostat 534 is a dynamicallyprogrammable thermostat and can be integrated with the control unit 510.For example, the dynamically programmable thermostat 534 can include thecontrol unit 510, e.g., as an internal component to the dynamicallyprogrammable thermostat 534. In addition, the control unit 510 can be agateway device that communicates with the dynamically programmablethermostat 534. In some implementations, the thermostat 534 iscontrolled via one or more property automation controls 522.

In some implementations, a module 537 is connected to one or morecomponents of an HVAC system associated with the property, and isconfigured to control operation of the one or more components of theHVAC system. In some implementations, the module 537 is also configuredto monitor energy consumption of the HVAC system components, forexample, by directly measuring the energy consumption of the HVAC systemcomponents or by estimating the energy usage of the one or more HVACsystem components based on detecting usage of components of the HVACsystem. The module 537 can communicate energy monitoring information andthe state of the HVAC system components to the thermostat 534 and cancontrol the one or more components of the HVAC system based on commandsreceived from the thermostat 534.

In some examples, the system 500 further includes one or more roboticdevices 590. The robotic devices 590 may be any type of robot that arecapable of moving and taking actions that assist in home monitoring. Forexample, the robotic devices 590 may include drones that are capable ofmoving throughout a property based on automated control technologyand/or user input control provided by a user. In this example, thedrones may be able to fly, roll, walk, or otherwise move about theproperty. The drones may include helicopter type devices (e.g., quadcopters), rolling helicopter type devices (e.g., roller copter devicesthat can fly and/or roll along the ground, walls, or ceiling) and landvehicle type devices (e.g., automated cars that drive around aproperty). In some cases, the robotic devices 590 may be robotic devices590 that are intended for other purposes and merely associated with thesystem 500 for use in appropriate circumstances. For instance, a roboticvacuum cleaner device may be associated with the monitoring system 500as one of the robotic devices 590 and may be controlled to take actionresponsive to monitoring system events.

In some examples, the robotic devices 590 automatically navigate withina property. In these examples, the robotic devices 590 include sensorsand control processors that guide movement of the robotic devices 590within the property. For instance, the robotic devices 590 may navigatewithin the property using one or more cameras, one or more proximitysensors, one or more gyroscopes, one or more accelerometers, one or moremagnetometers, a global positioning system (GPS) unit, an altimeter, oneor more sonar or laser sensors, and/or any other types of sensors thataid in navigation about a space. The robotic devices 590 may includecontrol processors that process output from the various sensors andcontrol the robotic devices 590 to move along a path that reaches thedesired destination and avoids obstacles. In this regard, the controlprocessors detect walls or other obstacles in the property and guidemovement of the robotic devices 590 in a manner that avoids the wallsand other obstacles.

In addition, the robotic devices 590 may store data that describesattributes of the property. For instance, the robotic devices 590 maystore a floorplan of a building on the property and/or athree-dimensional model of the property that enables the robotic devices590 to navigate the property. During initial configuration, the roboticdevices 590 may receive the data describing attributes of the property,determine a frame of reference to the data (e.g., a property orreference location in the property), and navigate the property based onthe frame of reference and the data describing attributes of theproperty. Further, initial configuration of the robotic devices 590 alsomay include learning of one or more navigation patterns in which a userprovides input to control the robotic devices 590 to perform a specificnavigation action (e.g., fly to an upstairs bedroom and spin aroundwhile capturing video and then return to a home charging base). In thisregard, the robotic devices 590 may learn and store the navigationpatterns such that the robotic devices 590 may automatically repeat thespecific navigation actions upon a later request.

In some examples, the robotic devices 590 may include data capture andrecording devices. In these examples, the robotic devices 590 mayinclude one or more cameras, one or more motion sensors, one or moremicrophones, one or more biometric data collection tools, one or moretemperature sensors, one or more humidity sensors, one or more air flowsensors, and/or any other types of sensors that may be useful incapturing monitoring data related to the property and users at theproperty. The one or more biometric data collection tools may beconfigured to collect biometric samples of a person in the property withor without contact of the person. For instance, the biometric datacollection tools may include a fingerprint scanner, a hair samplecollection tool, a skin cell collection tool, and/or any other tool thatallows the robotic devices 590 to take and store a biometric sample thatcan be used to identify the person (e.g., a biometric sample with DNAthat can be used for DNA testing).

In some implementations, one or more of the thermal cameras 530 may bemounted on one or more of the robotic devices 590.

In some implementations, the robotic devices 590 may include outputdevices. In these implementations, the robotic devices 590 may includeone or more displays, one or more speakers, and/or any type of outputdevices that allow the robotic devices 590 to communicate information toa nearby user.

The robotic devices 590 also may include a communication module thatenables the robotic devices 590 to communicate with the control unit510, each other, and/or other devices. The communication module may be awireless communication module that allows the robotic devices 590 tocommunicate wirelessly. For instance, the communication module may be aWi-Fi module that enables the robotic devices 590 to communicate over alocal wireless network at the property. The communication module furthermay be a 900 MHz wireless communication module that enables the roboticdevices 590 to communicate directly with the control unit 510. Othertypes of short-range wireless communication protocols, such asBluetooth, Bluetooth LE, Z-wave, Zigbee, etc., may be used to allow therobotic devices 590 to communicate with other devices in the property.In some implementations, the robotic devices 590 may communicate witheach other or with other devices of the system 500 through the network505.

The robotic devices 590 further may include processor and storagecapabilities. The robotic devices 590 may include any suitableprocessing devices that enable the robotic devices 590 to operateapplications and perform the actions described throughout thisdisclosure. In addition, the robotic devices 590 may include solid stateelectronic storage that enables the robotic devices 590 to storeapplications, configuration data, collected sensor data, and/or anyother type of information available to the robotic devices 590.

The robotic devices 590 can be associated with one or more chargingstations. The charging stations may be located at predefined home baseor reference locations at the property. The robotic devices 590 may beconfigured to navigate to the charging stations after completion oftasks needed to be performed for the monitoring system 500. Forinstance, after completion of a monitoring operation or upon instructionby the control unit 510, the robotic devices 590 may be configured toautomatically fly to and land on one of the charging stations. In thisregard, the robotic devices 590 may automatically maintain a fullycharged battery in a state in which the robotic devices 590 are readyfor use by the monitoring system 500.

The charging stations may be contact-based charging stations and/orwireless charging stations. For contact-based charging stations, therobotic devices 590 may have readily accessible points of contact thatthe robotic devices 590 are capable of positioning and mating with acorresponding contact on the charging station. For instance, ahelicopter type robotic device 590 may have an electronic contact on aportion of its landing gear that rests on and mates with an electronicpad of a charging station when the helicopter type robotic device 590lands on the charging station. The electronic contact on the roboticdevice 590 may include a cover that opens to expose the electroniccontact when the robotic device 590 is charging and closes to cover andinsulate the electronic contact when the robotic device is in operation.

For wireless charging stations, the robotic devices 590 may chargethrough a wireless exchange of power. In these cases, the roboticdevices 590 need only locate themselves closely enough to the wirelesscharging stations for the wireless exchange of power to occur. In thisregard, the positioning needed to land at a predefined home base orreference location in the property may be less precise than with acontact based charging station. Based on the robotic devices 590 landingat a wireless charging station, the wireless charging station outputs awireless signal that the robotic devices 590 receive and convert to apower signal that charges a battery maintained on the robotic devices590.

In some implementations, each of the robotic devices 590 has acorresponding and assigned charging station such that the number ofrobotic devices 590 equals the number of charging stations. In theseimplementations, the robotic devices 590 always navigate to the specificcharging station assigned to that robotic device. For instance, a firstrobotic device 590 may always use a first charging station and a secondrobotic device 590 may always use a second charging station.

In some examples, the robotic devices 590 may share charging stations.For instance, the robotic devices 590 may use one or more communitycharging stations that are capable of charging multiple robotic devices590. The community charging station may be configured to charge multiplerobotic devices 590 in parallel. The community charging station may beconfigured to charge multiple robotic devices 590 in serial such thatthe multiple robotic devices 590 take turns charging and, when fullycharged, return to a predefined home base or reference location in theproperty that is not associated with a charger. The number of communitycharging stations may be less than the number of robotic devices 590.

Also, the charging stations may not be assigned to specific roboticdevices 590 and may be capable of charging any of the robotic devices590. In this regard, the robotic devices 590 may use any suitable,unoccupied charging station when not in use. For instance, when one ofthe robotic devices 590 has completed an operation or is in need ofbattery charge, the control unit 510 references a stored table of theoccupancy status of each charging station and instructs the roboticdevice 590 to navigate to the nearest charging station that isunoccupied.

The system 500 further includes one or more integrated security devices580. The one or more integrated security devices may include any type ofdevice used to provide alerts based on received sensor data. Forinstance, the one or more control units 510 may provide one or morealerts to the one or more integrated security input/output devices 580.Additionally, the one or more control units 510 may receive one or moresensor data from the sensors 520 and determine whether to provide analert to the one or more integrated security input/output devices 580.

The sensors 520, the property automation controls 522, the thermalcamera 530, the thermostat 534, and the integrated security devices 580may communicate with the controller 512 over communication links 524,526, 528, 532, and 584. The communication links 524, 526, 528, 532, and584 may be a wired or wireless data pathway configured to transmitsignals from the sensors 520, the property automation controls 522, thethermal camera 530, the thermostat 534, and the integrated securitydevices 580 to the controller 512. The sensors 520, the propertyautomation controls 522, the thermal camera 530, the thermostat 534, andthe integrated security devices 580 may continuously transmit sensedvalues to the controller 512, periodically transmit sensed values to thecontroller 512, or transmit sensed values to the controller 512 inresponse to a change in a sensed value.

The communication links 524, 526, 528, 532, and 584 may include a localnetwork. The sensors 520, the property automation controls 522, thethermal camera 530, the thermostat 534, and the integrated securitydevices 580, and the controller 512 may exchange data and commands overthe local network. The local network may include 802.11 “Wi-Fi” wirelessEthernet (e.g., using low-power Wi-Fi chipsets), Z-Wave, Zigbee,Bluetooth, “Homeplug” or other “Powerline” networks that operate over ACwiring, and a Category 5 (CAT5) or Category 6 (CAT6) wired Ethernetnetwork. The local network may be a mesh network constructed based onthe devices connected to the mesh network.

The monitoring server 560 is one or more electronic devices configuredto provide monitoring services by exchanging electronic communicationswith the control unit 510, the one or more user devices 540 and 550, andthe central alarm station server 570 over the network 505. For example,the monitoring server 560 may be configured to monitor events (e.g.,alarm events) generated by the control unit 510. In this example, themonitoring server 560 may exchange electronic communications with thenetwork module 514 included in the control unit 510 to receiveinformation regarding events (e.g., alerts) detected by the control unit510. The monitoring server 560 also may receive information regardingevents (e.g., alerts) from the one or more user devices 540 and 550.

In some examples, the monitoring server 560 may route alert datareceived from the network module 514 or the one or more user devices 540and 550 to the central alarm station server 570. For example, themonitoring server 560 may transmit the alert data to the central alarmstation server 570 over the network 505.

The monitoring server 560 may store sensor data, thermal image data, andother monitoring system data received from the monitoring system andperform analysis of the sensor data, thermal image data, and othermonitoring system data received from the monitoring system. Based on theanalysis, the monitoring server 560 may communicate with and controlaspects of the control unit 510 or the one or more user devices 540 and550.

The monitoring server 560 may provide various monitoring services to thesystem 500. For example, the monitoring server 560 may analyze thesensor, thermal image, and other data to determine an activity patternof a resident of the property monitored by the system 500. In someimplementations, the monitoring server 560 may analyze the data foralarm conditions or may determine and perform actions at the property byissuing commands to one or more of the automation controls 522, possiblythrough the control unit 510.

The central alarm station server 570 is an electronic device configuredto provide alarm monitoring service by exchanging communications withthe control unit 510, the one or more mobile devices 540 and 550, andthe monitoring server 560 over the network 505. For example, the centralalarm station server 570 may be configured to monitor alerting eventsgenerated by the control unit 510. In this example, the central alarmstation server 570 may exchange communications with the network module514 included in the control unit 510 to receive information regardingalerting events detected by the control unit 510. The central alarmstation server 570 also may receive information regarding alertingevents from the one or more mobile devices 540 and 550 and/or themonitoring server 560.

The central alarm station server 570 is connected to multiple terminals572 and 574. The terminals 572 and 574 may be used by operators toprocess alerting events. For example, the central alarm station server570 may route alerting data to the terminals 572 and 574 to enable anoperator to process the alerting data. The terminals 572 and 574 mayinclude general-purpose computers (e.g., desktop personal computers,workstations, or laptop computers) that are configured to receivealerting data from a server in the central alarm station server 570 andrender a display of information based on the alerting data. Forinstance, the controller 512 may control the network module 514 totransmit, to the central alarm station server 570, alerting dataindicating that a sensor 520 detected motion from a motion sensor viathe sensors 520. The central alarm station server 570 may receive thealerting data and route the alerting data to the terminal 572 forprocessing by an operator associated with the terminal 572. The terminal572 may render a display to the operator that includes informationassociated with the alerting event (e.g., the lock sensor data, themotion sensor data, the contact sensor data, etc.) and the operator mayhandle the alerting event based on the displayed information.

In some implementations, the terminals 572 and 574 may be mobile devicesor devices designed for a specific function. Although FIG. 5 illustratestwo terminals for brevity, actual implementations may include more (and,perhaps, many more) terminals.

The one or more authorized user devices 540 and 550 are devices thathost and display user interfaces. For instance, the user device 540 is amobile device that hosts or runs one or more native applications (e.g.,the smart home application 542). The user device 540 may be a cellularphone or a non-cellular locally networked device with a display. Theuser device 540 may include a cell phone, a smart phone, a tablet PC, apersonal digital assistant (“PDA”), or any other portable deviceconfigured to communicate over a network and display information. Forexample, implementations may also include Blackberry-type devices (e.g.,as provided by Research in Motion), electronic organizers, iPhone-typedevices (e.g., as provided by Apple), iPod devices (e.g., as provided byApple) or other portable music players, other communication devices, andhandheld or portable electronic devices for gaming, communications,and/or data organization. The user device 540 may perform functionsunrelated to the monitoring system, such as placing personal telephonecalls, playing music, playing video, displaying pictures, browsing theInternet, maintaining an electronic calendar, etc.

The user device 540 includes a smart home application 542. The smarthome application 542 refers to a software/firmware program running onthe corresponding mobile device that enables the user interface andfeatures described throughout. The user device 540 may load or installthe smart home application 542 based on data received over a network ordata received from local media. The smart home application 542 runs onmobile devices platforms, such as iPhone, iPod touch, Blackberry, GoogleAndroid, Windows Mobile, etc. The smart home application 542 enables theuser device 540 to receive and process image and sensor data from themonitoring system.

The user device 550 may be a general-purpose computer (e.g., a desktoppersonal computer, a workstation, or a laptop computer) that isconfigured to communicate with the monitoring server 560 and/or thecontrol unit 510 over the network 505. The user device 550 may beconfigured to display a smart home user interface 552 that is generatedby the user device 550 or generated by the monitoring server 560. Forexample, the user device 550 may be configured to display a userinterface (e.g., a web page) provided by the monitoring server 560 thatenables a user to perceive images captured by the thermal camera 530and/or reports related to the monitoring system. Although FIG. 5illustrates two user devices for brevity, actual implementations mayinclude more (and, perhaps, many more) or fewer user devices.

The smart home application 542 and the smart home user interface 552 canallow a user to interface with the property monitoring system 500, forexample, allowing the user to view monitoring system settings, adjustmonitoring system parameters, customize monitoring system rules, andreceive and view monitoring system messages.

In some implementations, the one or more user devices 540 and 550communicate with and receive monitoring system data from the controlunit 510 using the communication link 538. For instance, the one or moreuser devices 540 and 550 may communicate with the control unit 510 usingvarious local wireless protocols such as Wi-Fi, Bluetooth, Z-wave,Zigbee, HomePlug (ethernet over power line), or wired protocols such asEthernet and USB, to connect the one or more user devices 540 and 550 tolocal security and automation equipment. The one or more user devices540 and 550 may connect locally to the monitoring system and its sensorsand other devices. The local connection may improve the speed of statusand control communications because communicating through the network 505with a remote server (e.g., the monitoring server 560) may besignificantly slower.

Although the one or more user devices 540 and 550 are shown ascommunicating with the control unit 510, the one or more user devices540 and 550 may communicate directly with the sensors 520 and otherdevices controlled by the control unit 510. In some implementations, theone or more user devices 540 and 550 replace the control unit 510 andperform the functions of the control unit 510 for local monitoring andlong range/offsite communication.

In other implementations, the one or more user devices 540 and 550receive monitoring system data captured by the control unit 510 throughthe network 505. The one or more user devices 540, 550 may receive thedata from the control unit 510 through the network 505 or the monitoringserver 560 may relay data received from the control unit 510 to the oneor more user devices 540 and 550 through the network 505. In thisregard, the monitoring server 560 may facilitate communication betweenthe one or more user devices 540 and 550 and the monitoring system 500.

In some implementations, the one or more user devices 540 and 550 may beconfigured to switch whether the one or more user devices 540 and 550communicate with the control unit 510 directly (e.g., through link 538)or through the monitoring server 560 (e.g., through network 505) basedon a location of the one or more user devices 540 and 550. For instance,when the one or more user devices 540 and 550 are located close to thecontrol unit 510 and in range to communicate directly with the controlunit 510, the one or more user devices 540 and 550 use directcommunication. When the one or more user devices 540 and 550 are locatedfar from the control unit 510 and not in range to communicate directlywith the control unit 510, the one or more user devices 540 and 550 usecommunication through the monitoring server 560.

Although the one or more user devices 540 and 550 are shown as beingconnected to the network 505, in some implementations, the one or moreuser devices 540 and 550 are not connected to the network 505. In theseimplementations, the one or more user devices 540 and 550 communicatedirectly with one or more of the monitoring system components and nonetwork (e.g., Internet) connection or reliance on remote servers isneeded.

In some implementations, the one or more user devices 540 and 550 areused in conjunction with only local sensors and/or local devices in ahouse. In these implementations, the system 500 includes the one or moreuser devices 540 and 550, the sensors 520, the property automationcontrols 522, the thermal camera 530, and the robotic devices 590. Theone or more user devices 540 and 550 receive data directly from thesensors 520, the property automation controls 522, the thermal camera530, and the robotic devices 590 (i.e., the monitoring systemcomponents) and sends data directly to the monitoring system components.The one or more user devices 540, 550 provide the appropriateinterfaces/processing to provide visual surveillance and reporting.

In other implementations, the system 500 further includes network 505and the sensors 520, the property automation controls 522, the thermalcamera 530, the thermostat 534, and the robotic devices 59 areconfigured to communicate sensor and image data to the one or more userdevices 540 and 550 over network 505 (e.g., the Internet, cellularnetwork, etc.). In yet another implementation, the sensors 520, theproperty automation controls 522, the thermal camera 530, the thermostat534, and the robotic devices 590 (or a component, such as abridge/router) are intelligent enough to change the communicationpathway from a direct local pathway when the one or more user devices540 and 550 are in close physical proximity to the sensors 520, theproperty automation controls 522, the thermal camera 530, the thermostat534, and the robotic devices 590 to a pathway over network 505 when theone or more user devices 540 and 550 are farther from the sensors 520,the property automation controls 522, the thermal camera 530, thethermostat 534, and the robotic devices 590. In some examples, thesystem leverages GPS information from the one or more user devices 540and 550 to determine whether the one or more user devices 540 and 550are close enough to the monitoring system components to use the directlocal pathway or whether the one or more user devices 540 and 550 arefar enough from the monitoring system components that the pathway overnetwork 505 is required. In other examples, the system leverages statuscommunications (e.g., pinging) between the one or more user devices 540and 550 and the sensors 520, the property automation controls 522, thethermal camera 530, the thermostat 534, and the robotic devices 590 todetermine whether communication using the direct local pathway ispossible. If communication using the direct local pathway is possible,the one or more user devices 540 and 550 communicate with the sensors520, the property automation controls 522, the thermal camera 530, thethermostat 534, and the robotic devices 590 using the direct localpathway. If communication using the direct local pathway is notpossible, the one or more user devices 540 and 550 communicate with themonitoring system components using the pathway over network 505.

In some implementations, the system 500 provides end users with accessto thermal images captured by the thermal camera 530 to aid in decisionmaking. The system 500 may transmit the thermal images captured by thethermal camera 530 over a wireless WAN network to the user devices 540and 550. Because transmission over a wireless WAN network may berelatively expensive, the system 500 can use several techniques toreduce costs while providing access to significant levels of usefulvisual information (e.g., compressing data, down-sampling data, sendingdata only over inexpensive LAN connections, or other techniques).

In some implementations, a state of the monitoring system and otherevents sensed by the monitoring system may be used to enable/disablevideo/image recording devices (e.g., the thermal camera 530 or othercameras of the system 500). In these implementations, the thermal camera530 may be set to capture thermal images on a periodic basis when thealarm system is armed in an “armed away” state, but set not to captureimages when the alarm system is armed in an “armed stay” or “unarmed”state. In addition, the thermal camera 530 may be triggered to begincapturing thermal images when the alarm system detects an event, such asan alarm event, a door-opening event for a door that leads to an areawithin a field of view of the thermal camera 530, or motion in the areawithin the field of view of the thermal camera 530. In otherimplementations, the thermal camera 530 may capture images continuously,but the captured images may be stored or transmitted over a network whenneeded.

The described systems, methods, and techniques may be implemented indigital electronic circuitry, computer hardware, firmware, software, orin combinations of these elements. Apparatus implementing thesetechniques may include appropriate input and output devices, a computerprocessor, and a computer program product tangibly embodied in amachine-readable storage device for execution by a programmableprocessor. A process implementing these techniques may be performed by aprogrammable processor executing a program of instructions to performdesired functions by operating on input data and generating appropriateoutput. The techniques may be implemented in one or more computerprograms that are executable on a programmable system including at leastone programmable processor coupled to receive data and instructionsfrom, and to transmit data and instructions to, a data storage system,at least one input device, and at least one output device. Each computerprogram may be implemented in a high-level procedural or object-orientedprogramming language, or in assembly or machine language if desired; andin any case, the language may be a compiled or interpreted language.Suitable processors include, by way of example, both general and specialpurpose microprocessors. Generally, a processor will receiveinstructions and data from a read-only memory and/or a random-accessmemory. Storage devices suitable for tangibly embodying computer programinstructions and data include all forms of non-volatile memory,including by way of example semiconductor memory devices, such asErasable Programmable Read-Only Memory (EPROM), Electrically ErasableProgrammable Read-Only Memory (EEPROM), and flash memory devices;magnetic disks such as internal hard disks and removable disks;magneto-optical disks; and Compact Disc Read-Only Memory (CD-ROM). Anyof the foregoing may be supplemented by, or incorporated in, speciallydesigned ASICs (application-specific integrated circuits).

It will be understood that various modifications may be made. Forexample, other useful implementations could be achieved if steps of thedisclosed techniques were performed in a different order and/or ifcomponents in the disclosed systems were combined in a different mannerand/or replaced or supplemented by other components. Accordingly, otherimplementations are within the scope of the disclosure.

What is claimed is:
 1. (canceled)
 2. A monitoring system comprising: athermal camera that is configured to generate a thermal image; and acomputer that is configured to: receive, from the thermal camera, thethermal image; based on the thermal image: detect a person or an animalwithin the thermal image, and determine a body temperature of the personor animal detected within the thermal image; determine whether the bodytemperature of the person or animal detected within the thermal imagesatisfies a temperature threshold; and based on the determination ofwhether the body temperature of the person or animal detected within thethermal image satisfies the temperature threshold, perform a monitoringsystem action.
 3. The system of claim 2, comprising: a sensor that isconfigured to generate sensor data that reflects an attribute of aproperty, wherein computer is configured to perform the monitoringsystem action based on the sensor data and the determination of whetherthe body temperature of the person or animal detected within the thermalimage satisfies the temperature threshold.
 4. The system of claim 2,wherein the thermal camera is mounted on a robotic device.
 5. The systemof claim 2, wherein the computer is configured to: detect the person oranimal within the thermal image by detecting a person within the thermalimage; determine the body temperature of the person or animal detectedwithin the thermal image by determining the body temperature of theperson detected within the thermal image; determine whether the bodytemperature of the person or animal detected within the thermal imagesatisfies the temperature threshold by determining whether the bodytemperature of the person detected within the thermal image satisfiesthe temperature threshold; and perform the monitoring system actionbased on the determination of whether the body temperature of the persondetected within the thermal image satisfies the temperature threshold.6. The system of claim 2, wherein the computer is configured to: detectthe person or animal within the thermal image by detecting an animalwithin the thermal image; determine the body temperature of the personor animal detected within the thermal image by determining the bodytemperature of the animal detected within the thermal image; determinewhether the body temperature of the person or animal detected within thethermal image satisfies the temperature threshold by determining whetherthe body temperature of the animal detected within the thermal imagesatisfies the temperature threshold; and perform the monitoring systemaction based on the determination of whether the body temperature of theanimal detected within the thermal image satisfies the temperaturethreshold.
 7. The system of claim 2, wherein the computer is configuredto perform the monitoring system action by adjusting one or more HVAC orthermostat settings based on the determination of whether the bodytemperature of the person or animal detected within the thermal imagesatisfies the temperature threshold.
 8. The system of claim 2, whereinthe computer is configured to perform the monitoring system action byperforming a monitoring system action that accounts for a particularcondition of a person detected within the thermal image.
 9. The systemof claim 2, wherein the computer is configured to determine the bodytemperature of the person or animal detected within the thermal image bydetermining an absolute body temperature of the person or animaldetected within the thermal image.
 10. The system of claim 2, whereinthe computer is configured to determine the body temperature of theperson or animal detected within the thermal image by determining arelative body temperature of the person or animal detected within thethermal image.
 11. The system of claim 2, wherein the computer isconfigured to identify a particular person in the thermal image throughfacial recognition.
 12. The system of claim 2, wherein the computer isconfigured to: detect the person or animal within the thermal image bydetecting multiple persons within the thermal image; and determine thebody temperature of the person or animal detected within the thermalimage by determining a body temperature distribution of the multiplepersons within the thermal image.
 13. A computer-implemented methodcomprising: receiving, by a monitoring system and from a thermal camera,a thermal image; based on the thermal image: detecting, by themonitoring system, a person or an animal within the thermal image, anddetermining, by the monitoring system, a body temperature of the personor animal detected within the thermal image; determining, by themonitoring system, whether the body temperature of the person or animaldetected within the thermal image satisfies a temperature threshold; andbased on the determination of whether the body temperature of the personor animal detected within the thermal image satisfies the temperaturethreshold, performing, by the monitoring system, a monitoring systemaction.
 14. The method of claim 13 wherein performing the monitoringsystem action comprises performing the monitoring system action based onsensor data captured by a sensor of the monitoring system and thedetermination of whether the body temperature of the person or animaldetected within the thermal image satisfies the temperature threshold.15. The method of claim 13, wherein: detecting the person or animalwithin the thermal image comprises detecting a person within the thermalimage; determining the body temperature of the person or animal detectedwithin the thermal image comprises determining the body temperature ofthe person detected within the thermal image; determining whether thebody temperature of the person or animal detected within the thermalimage satisfies the temperature threshold comprises determining whetherthe body temperature of the person detected within the thermal imagesatisfies the temperature threshold; and performing the monitoringsystem action comprises performing the monitoring system action based onthe determination of whether the body temperature of the person detectedwithin the thermal image satisfies the temperature threshold.
 16. Themethod of claim 13, wherein: detecting the person or animal within thethermal image comprises detecting an animal within the thermal image;determining the body temperature of the person or animal detected withinthe thermal image comprises determining the body temperature of theanimal detected within the thermal image; determining whether the bodytemperature of the person or animal detected within the thermal imagesatisfies the temperature threshold comprises determining whether thebody temperature of the animal detected within the thermal imagesatisfies the temperature threshold; and performing the monitoringsystem action comprises performing the monitoring system action based onthe determination of whether the body temperature of the animal detectedwithin the thermal image satisfies the temperature threshold.
 17. Themethod of claim 13, wherein performing the monitoring system actioncomprises adjusting one or more HVAC or thermostat settings based on thedetermination of whether the body temperature of the person or animaldetected within the thermal image satisfies the temperature threshold.18. The method of claim 13, wherein determining the body temperature ofthe person or animal detected within the thermal image comprisesdetermining an absolute body temperature of the person or animaldetected within the thermal image.
 19. The method of claim 13, whereindetermining the body temperature of the person or animal detected withinthe thermal image comprises determining a relative body temperature ofthe person or animal detected within the thermal image.
 20. The methodof claim 13, further comprising identifying a particular person in thethermal image through facial recognition.
 21. The method of claim 13,wherein: detecting the person or animal within the thermal imagecomprises detecting multiple persons within the thermal image; anddetermining the body temperature of the person or animal detected withinthe thermal image comprises determining a body temperature distributionof the multiple persons within the thermal image.